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Image sensing device and method 

Technical Field 

This invention relates to an image sensing device for 
5 digital image capture apparatus such as digital still 
cameras and analog and digital video cameras, scanners 
such as film or flat bed scanners, and other imaging 
systems and devices. 

10 Background of the Invention 

Unlike traditional cameras that use film to capture and 
store an image, digital cameras and other digital image 
capture devices as well as analog video cameras use a 
solid-state device, which is referred to herein as an 

15 image-sensing device to create an electronic 

representation of the image being captured. One type of 
image-sensing device which is in common usage is a mosaic 
type device in which an imaging photosensor array such as 
a Charge Coupled Device (CCD) , a Charge Injection device 

20 (CID) or a CMOS detector array is tiled with color filters 
in a Bayer pattern, in stripes or in some other regular 
arrangement. One example of such a prior art device is a 
Sony ICX2 05AK Progressive Scan CCD Image Sensor for Color 
Cameras. Image-sensing devices can contain millions of 

25 sampling sites, each having a photosensitive device or 
'sensor' and the photosensors being divided into a 
plurality of color channels. Each sensor records the 
intensity of the light that falls on it by converting 
photons into an electrical charge and accumulating the 

30 electrical charge over a fixed period of time. For each 
sensor, the collected charges are then processed into a 
signal which is subsequently digitized and the digital 
products saved in one of the many known digital image 
formats from which the image may be displayed, printed or 

35 further processed. 
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The photosensor's sensitivity to light varies as a 
function of wavelength. This sensitivity is usually 
adjusted to correspond to a particular color of light by 
means of a filter which selectively attenuates the light 
5 as a function of frequency. 

One commonly used system utilizes filters representative 
of three additive primary colors of red, green, and blue 
(RGB) , however other color systems are also known, 

10 including for example a system which utilizes filters of 
cyan, magenta, yellow and green. The selection of the 
colors comprising a set of primary colors is to some 
extent arbitrary and in theory many colors could be used. 
In a system whereby the colors are too broad, for example, 

15 cyan, magenta and yellow, the final computed RGB values 
generally suffer from an excess of noise. Conversely, if 
the colors are too narrow, the camera will have gaps in 
its sensitivity. For example, if red and green are very 
narrow, a yellow colored object which falls right between 

20 the passbands of the two filters may not be visible to the 
camera at all . 

Typical wavelength values of primary colors in an RGB 
system are 640nm (red) , 537nm (green) , and 464nm (blue) . 

25 

A well known technique used in many digital cameras for 
recording color images involves placing one color filter 
over each individual photosensor so that each photosensor 
can capture only one of the primary colors of the 
30 particular color system in use. For instance for an RGB 
system, repeating patterns of photosensors may be arranged 
such that each one of these photosensors has either a 
green, red, or blue filter placed to filter the light 
falling onto it. 

35 
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Some manufacturers of digital cameras have used even more 
than three different filter colors. For instance, Hewlett 
Packard has previously developed a camera that used four 
color channels with filters of cyan, magenta, yellow and 
5 green. Whilst it is common practice to convert from one 
set of colors to another (eg. from the sensor color system 
to a target color system) using a linear trans format ion, 
the conversion has the disadvantage of amplifying the 
signal noise. Furthermore, the amplification of noise 
10 increases when the difference between the target set of 
colors and the sensor set of colors increases. 

The majority of manufactures of digital cameras therefore 
use image sensor devices employing a three-color system in 

15 which each filter/photosensor combination has a profile 
that approximates the corresponding desired output target 
color. However this solution is problematic in that the 
target colors each have a narrow spectral bandwidth and as 
a result, the sensors do not record very much light. In 

20 some imaging applications a greater output is achieved by 
adopting filters associated with the photosensors which 
have spectral bandwidths that are more broadly tuned than 
would be ideal for optimal color rendition. Whilst this 
approach has the advantage of improving color sensitivity 

25 it also has the drawback of reducing the overall color 
quality. Furthermore, heightened sensitivity is only 
useful in shadow regions and generally causes saturation 
in highlighted areas of the resultant image. 

30 Another type of image sensing device which is sometimes 
seen in more expensive imaging systems employs a beam 
splitter to split the light delivered from a lens system 
into several paths each of which include a color filter 
and an image photosensor array. This approach avoids 

35 having to create a mosaic of filters in front of the 

photosensors, but introduces losses associated with the 
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beam splitter, is bulkier and requires one photosensor 
array for each of the beams emerging from the beam 
splitter. 

5 Prior art systems, where each color channel of an image 
sensing device contains sensors sensitive to a single 
spectral bandwidth, therefore suffer from either low 
saturation levels causing loss of detail in highlight 
areas and/or restricted sensitivity causing loss of detail 
10 in shadow areas and possibly further loss of detail where 
image colors fall between the spectral bandwidths of the 
respective color channels. 

Summary of the Present Invention 

15 Accordingly an image sensing device is provided 

comprising a plurality of photosensors arranged in at 
least one array, such that each of the photosensors 
converts incident light into an output signal, the 
photosensors and their respective output signals being 

20 divided into a plurality of color channels. A filter is 
associated with each of the photosensors, the filters 
selecting light within predetermined spectral bands for 
conversion by the photosensors into the output signals. 
One of the color channels is divided into at least two 

25 sub-channels and the filters associated with the 
photosensors of the at least two color sub-channels have 
overlapping spectral bands wherein one of the overlapping 
spectral bands is narrower in bandwidth than another of 
the overlapping spectral bands. 

30 

An image sensing device and a method of capturing an image 
will now be described by way of example, with reference to 
the accompanying drawings . 



200309636.1 



5 



Brief Description of the Drawings 

Fig. 1 is a schematic illustration of an image-sensing 
device with one portion shown in detail and in which a 
matrix of photosensors are provided arranged into four 
5 color channels or sub-channels; 

Fig. 2 is a schematic illustration of a portion of the 
image-processing region of a digital image capture device 
into which the image sensing device of Fig. 1 is 
10 incorporated; 

Fig. 3 is a schematic illustration of an alternative 
image-sensing device in which four arrays of photosensors 
are provided, each array associated with a respective 
15 color channel or sub-channel and light is distributed to 
each of the arrays by way of a beam splitter; 

Fig. 4 is a schematic illustration of a further 
alternative image-sensing device in which three arrays of 
20 photosensors are provided and light is distributed to each 
of the arrays by way of a beam splitter whereby two of the 
arrays are associated with a respective color channel and 
the third array is associated with two color sub-channels. 

25 Fig. 5 graphically illustrates a response function from a 
first and second color sensor of a single color channel 
(such as the green color channel of the embodiment of Fig. 
1) in an image sensing device; and 

30 Fig. 6 is a flow chart of a method of capturing an image. 

Detailed Description of Embodiments of the Present 
Invention 

Fig. 1 schematically illustrates a portion of an image- 
35 sensing device 100 showing in detail a subset 102 of the 
photosensors of the device and in particular a grouping 
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104 of four photosensors 106, 108, 110, 112 of the device. 
Each grouping 104 has a first color channel comprising a 
'red color photosensor' 106, a second color channel 
comprising a 'blue color photosensor' 108, and a third 
5 color channel comprising a first green color sub-channel 
having a first 'green color photosensor' 110 and a second 
green color sub-channel having a second 'green color 
photosensor' 112. Each of the four color photosensors, 
106, 108, 110 & 112 comprises a photodiode 114, 116, 118 & 

10 120 and filter 122, 124, 126 & 128 in combination to 

filter out all but the wanted wavelengths of the incident 
light and to convert the wanted wavelengths into output 
signals of the respective color channel or sub-channel. 
In the case of the green color channel, the first green 

15 filter/photosensor combination 110 has a filter 126 which 
is tuned to accept a broader band of wavelengths than the 
second green filter/photosensor combination 112 and 
therefore the first filter /photosensor combination has a 
higher sensitivity and is able to register lower levels of 

20 light than the second filter/ photosensor combination. In 
contrast the second green filter/photosensor combination 
112, because it is tuned to a narrower band of wavelengths 
than the first green filter/photosensor combination 110, 
has a lower sensitivity than the first filter/photosensor 

25 combination 126, 110, and is therefore less easily 
saturated. 

Image-sensing devices may, be integrated using 
architectures such as CCD, CID or CMOS architecture. The 

30 image-sensing device 100 may for example employ a sensor 
chip such as a Sony tm ICX2 05AL Progressive Scan CCD Image 
Sensor for B/W Cameras, to which a mosaic of primary color 
filters has been fitted. Alternatively the design of a 
device such as a Sony tm ICX205AK Progressive Scan CCD Image 

35 Sensor for Color Cameras may be modified to replace half 
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of the green filters in the in-built mosaic of filters 
with filters having a narrower acceptance bandwidth. 

The image sensing device may be incorporated in an image 
5 capture apparatus such as an analog or digital video 

camera, a digital still camera, a scanner such as film or 
flat bed scanner, or other imaging systems and devices. 

The filters 122, 124, 126, 128 may for example comprise :- 
10 1) Kodak tm Wratten* 11 #58 (green tricolor) for the 

first green filter 126 

2) Kodak tm Wratten tm #99 (green) for the second green 
filter 128 

3) Kodak tm Wratten tm #25 (red tricolor) for the red 
15 filter 122 

4) Kodak tm Wratten tm #47 (blue tricolor) for the blue 
filters 124 

Fig. 2 is a schematic illustration of a portion of the 

20 image-processing region of a digital image capture device 
into which the image-sensing device of Fig. 1 is 
incorporated. The image-sensing device in this example is 
a CCD 200. In accordance with CCD architecture a shift 
control circuit 202 controls the transport of an output 

25 signal from each photosensor location 2 04 to the next 
across the photosensor array 206. The output signals 208 
on the last row 210 of the photosensor array 206 are then 
transferred to the readout register 212. Once the signals 
in the last column 210 of photosensor locations 204 has 

30 been shifted to the readout register 212, the signals in 
the readout register are sequentially shifted into an 
analog to digital (A/D) converter 214 where they are 
digitized before being stored in digital memory 216 of a 
microprocessor 218. Once all of the pixels in the readout 

35 register 212 have been digitized and stored, the signals 
in the photosensor array 2 06 are again shifted by one 

200309636.1 



8 



photosensor location toward the readout register 212 such 
that the new signals in the last column 210 of photosensor 
locations 204 after the previous shift operation enter the 
readout register 212. This process continues until all of 
5 the signals in the photosensor array 2 06 have been read 
out. The captured and stored image is then available for 
further processing or for reconstruction for display on a 
display monitor or for printing. 

10 An alternative embodiment of an image sensing device is 
schematically illustrated in Fig. 3. In this embodiment, 
only a portion of which is shown, light typically enters 
the system 300 through a lens system 302 and is passed 
through a beam splitter 3 04 (eg. a series of prisms) which 

15 splits the light entering the imaging system into three 
color channels red, blue and green, where the green color 
channel has first and second sub-channels. Four image 
sensing devices 306, 312, 318 and 324 are provided, each 
comprising an array of photosensors, 310, 316, 322 and 

20 328, and a filter 308, 314, 320 and 326 where each filter 
covers the entire area of the respective photosensor 
array. Each array of photosensors 310, 316, 322 and 328 
may be implemented using a device such as the Sony tm 
ICX2 05AL Progressive Scan CCD Image Sensor for B/W Cameras 

25 or any similar device that does not include a mosaic of 
filters . 

Photosensors 310 and 316 of sensors 306 and 312 convert 
light in spectral bands that approximate the primary 

30 colors red and blue respectively. Photosensor 318 of 

image sensor 306 converts light in a first broad spectral 
band which approximates the primary color green whereas 
photosensor 328 of image sensor 324 converts light in a 
second narrower spectral band which also approximates the 

35 primary color green in a similar manner to the green sub- 
channels of the earlier embodiment. The photosensor 
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arrays are then each unloaded in a similar fashion to that 
of the previous embodiment and the image components are 
then combined in an image processor (not shown) . 

5 A further alternative embodiment of an image sensing 
device is schematically illustrated in Fig. 4, in which 
only a portion of the device is shown. 

In this embodiment, similar to the embodiment illustrated 
10 in Fig. 3, light enters the system 400 through a lens 

system 402 and is passed through a beam splitter 404 which 
splits the light entering the system into three color 
channels red, blue and green. In this embodiment, three 
image sensing devices 406, 412, and 418 are provided. 
15 Image sensing devices 406 and 412 each comprise an array 
of photosensors 410 and 416 and respective filters 408 and 
414 where each filter covers the entire area of the 
respective photosensor array. Photosensors 410 and 416 of 
sensors 406 and 412 convert light in spectral bands that 
20 approximate the primary colors red and blue respectively. 

Image sensing device 418 on the other hand comprises an 
array of photosensors 422 of which only a grouping of four 
photosensors 424 is shown in detail. Each grouping has a 

25 pair of first green color sensors diagonally spaced from 
one another, each having a photodiode 426 and a filter 428 
and a pair of second green color sensors each having a 
photodiode 430 and a filter 432. The filter 428 of the 
first green filter/photosensor combination is tuned to 

30 accept a broader rang of wavelengths than the second green 
filter/photosensor combination . 

The image-sensing device may comprise three or more color 
channels. In an alternative arrangement to those 
35 described above four color channels are provided where 
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each of the channels is indicative of the one of the 
colors cyan, magenta, yellow and green respectively. 

In a variation of the three color arrangements described 
5 above, the green color channel may comprise three or more 
sub-channels such that a filter of the first sub-channel 
is broadly tuned in spectrum, a filter of the second sub- 
channel is narrowly tuned in spectrum, and the filters of 
the remaining sub-channel ( s ) are tuned to bands between 
10 those of the first and second sensors. 



In particular applications it may for example be 
advantageous to provide two or more sensors for the red 
channel or the blue channel rather than the green channel . 

15 

Narrow band filters that may possible be used in a red 
sub-channel include Kodak tm Wratten tm 29 or 92 and narrow 
band filters that may possible be used in a blue sub- 
channel include Kodak tm Wratten tm 47B or 98. 

20 

In a still further example, more than one of the color 
channels and possibly all of the color channels may each 
comprise a plurality of sub-channels such that a filter 
of the first sub-channel for each respective color channel 
25 is broadly tuned in spectrum, the filter of the second 

sub-channel for each respective color channel is narrowly 
tuned in spectrum and the filters of any remaining sub- 
channels are tuned to bands between those of the first and 
second sub-channels of the respective color channel . 

30 

It should be appreciated that the invention is not limited 
to any particular combination of sensors and color 
channels and that the examples provided above are provided 
for illustrative purposes only. 

35 
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The graph illustrated in Fig. 5 shows output 
characteristics for two photosensors 502, 504 having 
filters with different spectral bandwidths (eg. Kodak^ 
Wratten tm #58 (green tricolor) and Kodak^ Wratten^ #99 
5 (green) ) associated with two sub-channels of a single 
color channel of a photosensor array. The filter of the 
first photosensor 502, is tuned to a broad spectral 
bandwidth, whilst the filter of the second photosensor 
504, is tuned to a narrow spectral bandwidth. 

10 

For low levels, or intensities of incident light on each 
photosensor 502, 504, the output of the sensor will not 
rise above an inherent noise floor and any useful signal 
is accordingly masked. Incident light levels falling at 

15 or below the noise floor are accordingly treated as black 
levels. On the other hand when high incident light levels 
fall on a sensor, the sensor is caused to saturate. 
Levels between these two extremes can be recorded by the 
photosensor as varying shades or tones between the black 

20 and saturation levels . 

Referring again to the graph of Fig. 5, the combined 
effect of the two photosensors 502 and 504 is to provide 
five different output areas depending on the intensity of 

25 the incident light that registers on each of the sensors. 
Area A represents darkness, where the signal generated by 
each of the sensors is unregistrable over the noise level. ' 
Area B represents areas of shadow detail, or regions of 
relatively low light intensity where only the photosensor 

30 502 with a broad spectral-band filter registers a signal 
raising above the noise floor. In Area B, the photosensor 
502 is able to register light because broad spectral 
bandwidth of the filter associated with the photosensor 
rejects fewer photons than that of the narrow-band 

35 photosensor 504. In contrast, the signal generated by the 
narrow spectral-band photosensor 504 still remains 
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unregistrable over the noise level because the narrower 
spectral -bandwidth of the filter associated with the 
photosensor rejects a larger proportion of the incident 
photons. Area C represents mid-tones, where the intensity 
5 of light incident on each of the photosensors 502 and 504 
is sufficient for them to generate a signal that is above 
the noise floor. The fourth Area, D represents areas of 
highlight detail, or regions of relatively high light 
intensity where only the photosensor 504 with a narrow 

10 spectral-band filter registers a signal below the 

saturation level. In Area D, the photosensor 504 is able 
to register a non-saturated signal because the narrow 
spectral-bandwidth of the filter associated with the 
photosensor rejects a larger proportion of incident 

15 photons than the broad-band photosensor 502 and is 
therefore less easily driven into saturation. In 
contrast, at this intensity of incident light, the broad 
spectral-band filter of photosensor 502 is passing more 
photons than are required to drive the output of the 

20 photosensor into saturation. Finally, area E represents 
full highlight, where both of the photosensors have been 
driven into saturation. By suitably scaling and combining 
the output signals ,of the two photosensors 502 and 504 it 
is possible to generate a range of recorded light 

25 intensities within a given color channel which greatly 
exceed those that can be captured with a single 
photosensor. Employing more than two photosensors per 
color channel can further extend the range. 

30 In comparison with prior art systems where each color 
channel of an image sensing device contains sensors 
sensitive to a single spectral band, this image sensing 
device has the advantage of improved dynamic range or 
exposure latitude and offers the possibility of improved 

35 sensitivity providing greater detail in shadow areas 

and/or higher saturation levels giving improved detail in 
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highlight areas. As illustrated in Fig. 5, the 
combination of dual sensors for a single color provide a 
useful output over a greater range of input intensities 
than will be the case for a single sensor. 

5 

A method of capturing an electronic representation of an 
image will now be described. In a first step of the 
method, the image is projected onto a sensor device 
comprising a plurality of photosensors. The wavelengths 

10 of light incident on each photosensor are restricted to a 
spectral band defining a color associated with the color 
channel of the respective photosensor. The output of each 
photosensor is therefore a measurement of the intensity of 
light incident on the respective photosensor. The outputs 

15 of the photosensors are then combined to generate an 

electronic representation of the image. One color channel 
is divided into at least two sub-channels having 
overlapping spectral bands wherein one of the overlapping 
spectral bands is narrower in bandwidth than another of 

20 the overlapping spectral bands . 

Referring to Fig. 6 a flow chart 600 is illustrated 
showing the steps of the image capturing method. The 
first step 502 in any imaging process is to project the 

25 image onto the sensor device. This is typically performed 
by using a lens system to focus light from a scene (or in 
the case of a scanning system from a document or item to 
be scanned) onto the image sensor device. Depending upon 
the type of imaging system in use, the projected image is 

30 distributed 604 to sensors associated with different color 
channels, either by mixing the individual photosensors in 
a single photosensor array, or by splitting the projected 
image into a number of beams (using a beam splitter) and 
using a separate array to detect each color channel. In 

35 either case a filter is used in the light path to each 
photosensor to restrict 606 the wavelengths of light 
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incident on each photosensor to only those wavelengths 
associated with the color channel of the photosensor. In 
the case of the light splitter approach a single filter 
element may be employed over each photosensor array, 
5 whereas in the case where photosensors of a given color 
channel are spatially distributed with those of other 
channels in a single array, a mosaic of filters will be 
employed over the array. However in each case at least 
one color channel is divided into sub-channels including 

10 at least one sub-channel tuned to record a broad band of 
wavelengths and one tuned to record a narrow band of 
wavelengths. In the example given above the green channel 
is divided into two sub-channels using respectively a 
Kodak tm Wratten^ #58 (green tricolor) filter to select a 

15 broad band of green wavelengths and a Kodak tm Wratten tm #99 
(green) to select a narrow band of green wavelengths. On 
the other hand the Red and Blue channels may use 
respectively a Kodak tm Wratten tm #25 (red tricolor) to 
select a broad band of red wavelengths and a Kodak tm 

20 Wratten tm #47 (blue tricolor) to select a broad band of 
blue wavelengths . 

The intensity of light falling on each photosensor is 
measured using one or more sensor arrays of a type such as 

25 the Sony ICX2 05AL and digitized using a suitable analog to 
digital converter. For the green channel, and/or any 
other channel that has multiple sub-channels, the sub- 
channel signals are scaled and extended 610 by 
interpolation of signals from the other sub-channels of 

30 the same color channel. Therefore, in the case of those 
areas of input intensity where one of the sub-channel 
sensors is producing an output that is either at the black 
level or the saturation level, the suitably scaled outputs 
of other photosensors of the same color channel are used 

35 to interpolate a signal for the photosensor that has an 
output at the black level or the saturation level. 
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Finally the color channel and sub channel signals are 
color corrected 612 using, for example, 4x3 color 
correction matrix, to produce a digital image with three 
5 channels of color such as an Srgb image. The color 

correction step is also known as demosaicing, and is used 
to convert the raw image data from the photosensor array 
into a resultant standard image format; that is, 
calculating the red, green, and blue intensities of 

10 nominal pixel locations of the resultant image from the 
photosensor array data. Since respective color channels 
of the photosensor array may not be aligned to a 
rectangular sampling geometry, an algorithm such as that 
proposed by David Taubman may be utilized (Taubman, David. 

15 Generalized wiener reconstruction of images from color 

sensor data using a scale invariant prior, Proceedings of 
the 2000 International Conference on Image Processing 
(ICIP 2000) , 10-13 September 2000) . A variation of this 
algorithm may be employed to allow for the multiple sub- 

20 channels within a single color channel. 

Thus various embodiments and components have been shown 
for creating an image sensing system, and these may be 
used singly or in combination, or with other elements 

25 known in the arts of optical design, and color science. It 
is understood that these and other such combinations, 
substitutions, and alternative embodiments may be 
undertaken according to the requirements and materials at 
hand without deviating from the spirit of the invention, 

30 the scope of which is to be limited only by the claims 
appended hereto. 
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